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This urban transportation pamphlet delves into the 
roles of policy groups and technical staffs in evaluating alternative 
transportation plans, evaluation criteria, systems to evaluate, and 
evaluation procedures* The introduction admits the importance of 
subjective, but informed, judgment as an effective tool in weighing 
alternative transportation plans, together with evaluation of roles, 
procedures, goals, costs, employment, land utilization, the tax base, 
accessibility, and social considerations The environment is stressed 
throughout the pamphlet* "Out of this vast collection of data aust 
emerge those items of data necessary for a group of normally 
nontechnical, policy-oriented individuals to render intelligent 
judgments concerning the relative merits of plan alternatives, n 
states the conclusion* Charts, graphs, and tables are liberally used. 
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Evaluation of Alternative Ti asportation Systems 



INTRODUCTION ("Evaluation J To Examine and Judge") 

The urban transportation planning process may be viewed as a 
vehicle for providing rational information to decisionmakers 
and the public concerning the allocation of transportation 
resources. One of the most basic and widely accepted means for 
conveying this information is the long range regional transpor- 
tation plan. The development of a comprehensive long range trans- 
portation plan normally consumes a large share, of the manpower 
and budget of an urban transportation planning program. 
Representing a current conception of the regional transportation 
system at some point in the distant future (20 to 25 years), the 
long range plan should serve as the basis for key transportation 
improvement decisions within a metropolitan area, deluding the 
development of sho::t range improvennnt programs, annual capital 
budgets, and project planning. 

The adoption of a long range plan to guide investments in urban 
transportation facilities normally occurs upon the completion of 
an orderly process of developing and testing a wide range of alter- 
native future land use/ transportation configurations, narrowing 
the possibilities to a relatively small number of basic alternatives 
and performing an in depth evaluation of each to determine which plan 
can best serve community objectives. The process of plan evalua- 
tion can be quite complex, due not only to the almost infinite 
variety of possible alternatives, but perhaps more significantly 
due to the vast array of possible criteria by which plans may be 
judged. Many of these evaluation criteria are quite subjective and 
difficult to forecast with accuracy. Yet, increasingly , it is 
the more subjective and elusive criteria that are having the 
strongest influence on transportation related decisions. 

Techniques have, arisen within the past de:ade which attempt to 
accommodate these seemingly incompatible mixes of evaluation 
criteria by applying ranking, weighting, and scaling mechanisms. 
While useful as tools for relating a multiplicity of diverse 
factors, those techniques are limited in terms of thier ability 
to overcome the need for subjective judgment. More and more, 
it is beginning to appear that subjective, but informed , judg- 
ment will be recognized as the effective tool it always has 
been in weighing alternatives, making the necessary tradeoffs 
and performing the evaluations required for the rational selec- 
tion and adoption of a land use/ transportation plan. 



2 



II. THE ROLES OF POLICY CROUPS AND TECHNICAL STAFFS IN EVALUATING 
ALTERNATIVE PLANS ("Not to Decide is to Decide") 

It is unfortunate that there aru many who view transportation 
system evaluation as just another technical element of the urban 
transport tion planning process, when in fact, the role of the 
policy maker should be far stronger than that of the technician 
in this phase of transportation planning* The flox* diagram 
shown in figure 1 is offered as an example of the major role to 
be played by elected officials and their representatives in the 
developing, testing, evaluating, and adopting of transportation/ 
land use plans • At the risk of oversimplifying a complex 
political and technical process, the illustration points out 
that virtually every step requires some degree of input from 
a policy group, ranging from the formulation of alternatives 
for initial testing tluough the adoption of the long range plan 
and program. 

A, Policy Group Role 

The role of the policy group which oversees and guides the 
overall planning effort is basic to the evaluation processo 
It includes the following major activities: 

1, Adopt Regional Goals and Objectives 

The development of a viable planning guide for future 
growth rests upon a foundation of knowledge of the 
community's aspirations and needs. It is the responsi- 
bility of a policy group to see to it that the necessary 
studies and programs are conducted to enable the develop- 
ment of a set of community goals and objectives. It is 
further the responsibility of this policy group, acting 
as representatives of the various interests within the 
community, to adopt meaningful goals and objectives to 
guide the development of alternative plans, 

2. Approve Alternatives for Testing and Evaluation 

The decision as to which alternatives should be subject 
to initial testing, and which to indepth evaluation should 
not be left to the sole discretion of the technical staffo 
While professional staff members may provide information 
and advice concerning possible alternatives, the responsi- 
bility of the policy group for ultimate plan adoption 
renders as virtually self-evident the need for this same 
group to subscribe to the variety of alternatives to be 
examined. 
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• ADOPT REGIONAL GOALS AND OBJECTIVES 

• APPROVE ALTERNATIVES FOR TESTING AND 

EVALUATION 

• SELECT EVALUATION APPROACH AND 

APPROVE EVALUATION PROCESS 

• DESIGNATE MECHANISMS FOR CITIZEN 

INVOLVEMENT 

• WEIGH ALTERNATIVES AND SELECT PLAN 

FOR ADOPTION 

• ADOPT LONG RANGE PLAN AND SHORT 

RANGE PROGRAM 



Select Evaluation Approach and Approve Evaluation 
Process 

As will be evident later in this paper, the nature of 
the evaluation process is such that it may be strongly 
influenced by the particular set of techniques chosen. 
It is therefore quite important that the group expected 
to reach a decision should have a strong voice in 
accepting, rejecting, or modifying to suit its needs, 
the particular method by which the decision is to be 
reached. If the evaluation procedure is selected by the 
professional staff, it should be with the full awareness 
and approval of the policy group. 

Designate Mechanism for Citizen Involvement 

One of the more critical issues confronting the planning 
profession concerns the question of community involvement. 
Although the determination of effective means of providing 
for citizen participation in developing long range plans 
remains a perplexing problem for members of technical 
staffs, it must be remembered that government agencias 
exist to implement the programs and decisions of elested 
officials, and that any mechanism de\ sloped for providing 
a greater degree of citizen participation must have the 
official blessing of these decisionmakers. 

Weigh Alternatives and Select Plan for Adoption 

Quite frequently the efforts of a technical staff in 
developing information concerning the attributes of plan 
alternatives are mistakenly considered ;> represent the 
entire evaluation process. In fact, the evaluation of 
alternatives and selection of a plan represent actions 
which are largely the responsibility of the policy group. 
While it is the role of the staff to provide information 
necessary for the evaluation of alternatives, the assimi- 
lation of this information, the weighing of advantages 
and disadvantages, and the ultimate choice of a single 
"best" plan should take place within a policy forum. 

Adoption of Long Range Plan and Short Range Program 

The fruition of the evaluation process consists of an 
action by tho policy bedy toward formal adoption of the 
long range plan, and a short range improvement program 



based upon the plan* With this degree of acceptance 
of the plan, it should become the standing policy of 
all participating governments and agencies to adhere 
to the plan and program in implementing transportation 
improvements. At such time as they are no longer useful 
nor relevant in guiding regional transportation decisions, 
the plan and program should undergo redevelopment under 
the direction of the policy group* 

Staff Role 

It is fundamental to the concept of government planning in 
a democratic society that the role of the professional 
planner is not to adopt and implement plans, but to provide 
the necessary information and advice for elected officials 
and their representatives to reach these types of decisions. 
The following represents some of the basic functions of the 
professional planning staff in evaluating transportation and 
land use alternatives. 

1. Develop Technical Procedures 

Perhaps the key role to be played by a professional 
planning staff involves the development and application 
of technical procedures capable of accounting for the 
complex phenomenon avssoeiated with urban development 
and transportation systems dynamics. It is at this 
point that the planner demonstrates why his skill and 
training allow him to provide information and data to 
decisionmakers which would be available from no other 
source, and which are uniquely tailored to the noeds of 
the decisionmaker and the community. 

It is essential that the professional planner take time 
to communicate with his sponsors in terms comprehensible 
to them, generally hov/ his techniques worh, what assump- 
tions he has made, and why the information he has 
developed should be used in reaching decisions. 

2. Recommend System Alternatives to Test and Evaluate 

While the responsibility for final designation of alter- 
native plans for testing and evaluating remains with a 
policy oriented body, the experience and professional 
wisdom of the technical staff is an invaluable resource 
in suggesting possibilities, and in generally broadening 
the horizons of policy groups. 
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• DEVELOP TECHNICAL PROCEDURES 



•RECOMMEND SYSTEM ALTERNATIVES 
TO TEST AND EVALUATE 



• SUGGEST POSSIBLE EVALUATION 
APPROACHES TO POLICY GROUP 



• PROVIDE NECESSARY DECISION 
INFORMATION 



• COMMUNICATE WITH PRIVATE CITIZENS 
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Suggest Possible! Evaluation Approaches to Policy Group 

Since Che selection of an evaluation procedure can have 
an important hearing upon the entire evaluation process, 
it is important that the staff delineate clearly ?or the 
policy group specifically what evaluation procedures are 
available, and what are the strengths and weaknesses of 
each. While the staff may .suggest an approach, the 
selection of an evaluation technique ought to bo subject 
to the approval of the policy body. 

Provide Necessary Decision Information 

'Hie exercising of evaluation techniques by a technical 
staff should be thought of not as a decision technique, 
but as an Information generating function. Once again, 
the staff's role is to provide the information required by 
a policy group to reach decisions. This information should 
be clear, concise, timely, relevant, and specifically 
tailored to each unique decision requirement. Conveying 
effective and useful information to policy groups, as 
well as to the public, is a difficult process which most 
technical staffs have yet to master. 

Communicate With Private Citizens 

As indicated earlier, developing effective means for com- 
munication with, and participation of private citizens 
remains one of the planners most difficult and frustrating 
problems. Within the context of ground rules established 
by a policy group, the professional staff faces the chal- 
lenges of (a) identifying the multiplicity of citizen 
interests within the community, (b) identifying credible 
spokesmen from the community, (c) arousing citizen interest 
in the development of n long range regional plan, and 
(d) translating the often conflicting inputs gained from 
the community into plan alternatives which are generally 
acceptable to both policy makers and the community. 

Recommend a Plan and Short Range Program 

While at this poin: it should be apparent that the selec- 
tion and adoption of a plan and program are the responsi- 
bility of the policy group, the wealth of knowledge 
available to a professional staff, in conjunction with the 
planner's ability to play an objective and even handed 
role, makes ic quite desirable, as a rule, for the planning 
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PARTICIPATE IN THE FORMULATION OF 
REGIONAL GOALS, OBJECTIVES, AND 
EVALUATION CRITERIA 

PARTICIPATE IN THE DEVELOPMENT OF 
ALTERNATIVE PLANS 

COMMUNICATE WITH POLICY GROUP AND 
TECHNICAL STAFF MEMBERS 
PARTICIPATE IN THE EVALUATION PROCESS 
PARTICIPATE IN THE PLAN SELECTION ACTION 

MAKE BOTH NEGATIVE AND POSITIVE 
OPINIONS KNOWN TO DECISION-MAKERS 
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staff to reach some sort of recommendation as to which 
alternative it fools should be adopted. Naturally, the 
degree to which a staff position is heeded, or even 
solicited, depends entirely upon the unique circum- 
stances of each individual transportation planning 
program* 

III. EVALUATION CRITERIA ("It Is Impossible to Judge a Book By its 

Cover Alone") 

The evaluation of alternative transportation plans is a process 
which attempts to measure the ability of plans to achieve stated 
transportation goals and objectives. It, therefore, U necessary 
to delineate transportation goals and objectives within the frame- 
work of more general development objectives before the evaluation 
process begins. Actually, goals and objectives should be developed 
early in the planning process and used in plan formulation as 
well as in evaluation. 

The goals of a region should be those of the people and not of the 
planning staff. It is, therefore, desirable to involve the largest 
possible number of people in their formulation. Policy committees, 
technical committees, citizen committees, and conmittees composed 
of the power structure of the community have been used success- 
fully in this area. Some studies have taken surveys, sponsored 
discussion groups, or held public hearings to assist in developing 
community goals and objectives. It is important that these goals 
and objectives be formally adopted by the policy making bodies of 
the region. 

Evaluation criteria represent the standards by which alternative 
plans are judged in terms of their ability to satisfy auopted 
goals and objectives. As such, they ate the basic yard sticks 
for measuring those attributes of each alternative which have 
been identified as having a strong bearing on the ultimate selec- 
tion of a plan. As stated earlier, it is the responsibility of 
the professional staff to indicate possible criteria in each 
goal area, to develop the necessary information in an objective 
and reliable way, and to furnish this information to policy groups 
in a comprehensible manner. The final selection of evaluating 
criteria, however, should be made by the policy group after it 
is fully briefed as to what possibilities exist. 

As an example of the process of developing an acceptable set of 
evaluation criteria, consider a regional goal to minimize dis- 
ruption caused by transportation improvements, and an associated 
objective U keep the number of dwelling units taken for right- 
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of -way acquisition bo low a stated maximum. In developing 
evaluation criteria to measure how well each alternative meets 
such a goal and objective, the most obvious becomes the "number 
of dwelling units taken," There are other ways of measuring 
and analyzing dislocations , however, and these should be explained 
carefully to the policy group, For example, dislocations can be 
measured in terms of percent distributions by race, or by income 
group, or by political jurisdiction. The estimated number of 
dislocations associated with each alternative can be subtracted 
from the number of joint development opportunities for additional 
housing. Finally, the number of dislocations per year can be 
compared to historical annual rates to lend a better scale, and 
greater meaning to the numbers being considered. 

Thus, for any particular objective, there normally exists a large 
variety of possible evaluation criteria. Each individual cri- 
terion, however, is normally quite limited in its ability to 
provide a complete indication of how well a particular plan may 
satisfy a stated objective. Furthermore, criteria should be 
couched in terms which lend meaning to the often unfathomable 
array of numbers produced in a plan evaluation process. 

There are three basic types of evaluation criteria; cos table, 
quantifiable, and qualitative. They may be defined as follows: 

Costable Criteria - Those criteria which can be 
measured in terms of dollar costs and benefits. 

Quantifiable Criteria - Those criteria which may 
be measured directly in terms of numerical 
quantities. 

Qualitative Criteria - Those criteria which may he 
described directly only according to subjective 
means (but may ultimately be quantified somewhat 
through ranking and scaling techniques ). 

Figure 2 shows several examples of each type of evaluation 
criteria, as they reflect system user, and external attributes 
of each plan alternative. The following discussion delves a 
little more deeply into a few of the more commonly applied 
evaluation criteria. 

A. Total Transportation Cost 

The past several years has witnessed a noticeable shift 
away from the use of costable factors as the sole criteria 
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Figure 2 - Examples of Evaluation Criteria 



Type of 
Attribute 


Type of Criteria 


Cos table 1 Quantifiable 


Qualitative 


i 

System j Construction 
i Right-of-Way 
1 Transit Vehicles 
Maintenance 
Transit Operations 

1 

i 


Miles of System by 
Class and Location 

Transit Seat Miles 

Average Freeway 
Spacing 

TTAnfi'ffc Vehicle 

Headways 



Aesthetics 


! 1 1 

User ! Private Vehicle Operation 
j Accident Costs 
| Travel time Costs 
1 (Through Value of Time) 
! Fares 
i 


Accessibility Measures 
Vehicle-Miles of Travel 
Percent Transit Use 
Trip Length 
Effective Speed 


Comfort 
Convenience 
Visual Aspects 
from the Road 


External 


Change in Property Values 
| Ta:: Base 

j Extra Relocation Payments 
j Construction of 

Replacement Housing 


Air Pollution 
Number of Dislocations 
Land Consumption 
i Noise 


Neighborhood 
Disruption 

Psychological 
Attitudes 
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for plan evaluation. The trend has been towards a more 
balanced view, with costable criteria becoming just one 
member of a broader group of criteria. Nevertheless, the 
ability to weigh the sosts of each alternative remains one 
of the planners most reliable and accepted ways of judging 
the effectiveness of alternative plans. 

A particularly useful approach toward balancing the dollar 
costs and benefits of alternative plans was applied 15 years 
ago in the original Chicago Area Transportation Study. The 
technique illustrated in figure 3 recognizes that virtually- 
all costs of transportation service must ultimately be born 
by members of the public, both in the form of direct costs 
involved in using transportation facilities, and in the less 
direct way of paying taxes which are ultimately used to pro- 
vide transportation facilities. As shown in figure 3, the 
total transportation cost is therefore the sum of the costs 
associated with the use of transportation facilities and 
the cost of providing these improvements. Quite naturally, 
the greater the degree of transportation improvement, the 
higher the costs of providing the Improvements. On the 
other hand, as the degree of improvement increases, the costs 
of using the facilities (such as operating, traveltime, and 
accident costs) decrease. However, the rate at which user 
costs decline is itself decreasing. Thus, the first "X" 
dollars spent for transportation improvements, if invested 
wisely, will produce a greater reduction in user costs than 
the second "X" dollars, and so on. 

From figure 3, it can be seen that the total transportation 
cost, which is the sum of the costs of providing capacity 
and the costs of using the facilities, reaches a minimum 
point. This minimum point, by definition, occurs when an 
added dollar toward providing capacity produces less than a 
dollar savings in user costs. Naturally, this economic 
analysis is performed using appropriate economic techniques 
to account for both the variation in costs over the 20- to 
25-year time horizon as well as the time value of money, 
or discount rate. 

Accessibility 

The concept of accessibility has been developed as a basic 
indicator of the ability of a transportation system to pro- 
vide efficient srvice among areas of activity within a 
planning region. There are numerous ways of measuring and 
displaying accessibility indicators. These include the 
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j . Example of Total Transportation Cost Evaluation 

Criterion 
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■ENVIRONMENTAL MATRIX 
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• CASE STUDIES 



A COORDINATED CONSIDERATION OF NOISE IMPACTS 
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development of mathematical indices (such as the denominator 
of the gravity model) and the use of graphical displays (auch 
as isochronal maps and data plots). The key questions to be 
considered in analyzing accessibilities among and within 
alternative transportation plans concern: 

1. Accessibility for which groups (low income, high income, 
central city, suburbs, etc.). 

2. Accessibility to what activities (employment, shopping, 
social, health services, etc.). 

3. Accessibility by. which mode of transportation (transit, 
private vehicles, commerical vehicles, etc.). 

4. Accessibility during what time of day (peak, off-peak). 

Figures 4 and 5 and table 1 illustrate three techniques for 
viewing accessibility. Figure 4 shows a graph which depicts 
the percentage of unskilled jobs within a region which can 
be reached by low income residents of a particular district 
under each of two alternative plans. Under this example, 
Plan B is superior to Plan A in terms of providing low 
income residents of District I with peak hour public trans- 
portation access to suitable jobs. Similar graphs may be 
developed for any number of combinations of accessibilities 
and incorporated into the overall plan evaluation process at 
the discretion of the staff and policy group. 

Figure 5 depicts the use of an isochronal map to reflect 
relative traveltimes from one part of a region to all other 
parts. An isochronal map is simply a map showing lines which 
connect points of equal travel time from a particular locality. 
(The lines are called isochronals, or traveltime contours.) 
The same types of stratifications used in figure 4 are, of 
course, possible here as well. 

Table 1 illustrates yet another measure of accessibility, i.e., 
effective speed. Effective speed simply involves measuring 
the airline distance covered in a particular time period 
spent traveling between two points. The underlying concept 
is that people, if they were able to do so, would travel 
between points in a straight line, or airline distance. The 
time required to cover this distance is a measure of the ser- 
vice provided by a transportation system. The data shown in 
table 1 show effective speeds for work trips stratified by 
income group, auto availability, and location. 




fir; 



f . . , ; p e ^ k * Tj-aveXtime (Minutes) From District 1 
T I (Low Income) Via Public Transportation 



r-^ 



.4^-4 



Hgure 4 



' Example of Accessibility Data Plot.., 

— 4-f. * 



842 




LINE OF CONSTANT 
TRAVELTIME 



X 

2ont:. . m 

Trip Purpott:,. Wo ?ft , 



»y ■ 1 — 



Numbtr cf Productions; 



C6NTROIO AND 
NUMBER OF 
ATTRACTIONS 
IN EACH ZONE 



FIGURE 5 TYPICAL ISOCHRONAL MAP 



ML-::: 



i:r: 



Source: "Methods of Evaluation of -he Effects of Trai. portatioti Systems 
on Coamunity Values," SRI, 1971, ... . 'I ... . j ' I'M" !. J 

- • r • : - -— \z ~- h - 1~- 



bk; ioxio 



9 

ERIC 



• 



14 



Table 1 



Average Miles* Covered Per Ten Minutes of Work Trip Traveltime 



Household Income 



Residential 
Location 



Autos 
Available 



Under 


$4,000 - 


Over 


$4,000 


$10,000 


$10,000 


1.0 


1.2 


1.4 


1.4 


1.6 


1.7 


1.0 


1.6 


1.9 


1.9 


2.5 


3.1 



New York City 
(excluding Richmond) 

Outside New York City 
(including Richmond) 



0 

1+ 
0 

1+ 



Airline Distance 
Source : 



H. Kassoff and H. Deutschman, "Transportation: The Link 
Between People and Jobs, " Highway Research Record 322. 

C. Analysis of External Effects 

The degree to which so-called "external" consequences of trans- 
portation improvements have grown importance is reflected by 
the major concern over disruption and dislocations caused by 
large scale transportation improvements. Table 2 and figure 6 
illustrate techniques for developing and displaying such 
information for the purposes of plan evaluation. 

Table 2 simply depicts the numbers of residences dislocated 
for each of five alternatives (including the existing plus 
committed). The important point to be gained from this table 
is the need for disaggregating gross statistics into meaningful 
stratifications. For example, total estimated residential 
dislocations tell only part of the story. By stratifying 
according to income group, or race, or location, much can be 
learned in terms of possible inequitable hardships that would 
be masked by treating only aggregate statistics. 
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Table 2 ' 



^— Sample Analysis of Residential Dislocations- 
. ■ , by Incomo Group .'. 




• 


Plan 
Alternative 


dumber of 
Dwellings Dislocated 
(By Income Group) 


Total 

uwei lingo 




Low 


Middle 


Hi£h 


Dislocated ' 




Bxi sting 
'+ Committed 


35 


60 


5 


150: 




a : ~i 


400 


75 


' *5 ' 


480 






250 




~15 


390 






ISO 


175 


50 


375 




. . . * 

D 


75 


175 


75 


325 



I.... 



4 — . 

-i ■ • 



... r 



i 



• ♦ t — - 



. * .... 

XL. 



'■'A 

■ -I 



..... iii. 



■ r 



Number 
of 

1 Annual 
• -Dislocations 



•■ t 



60 



40 



•— I ■ • 



20 



1 • Jt"' 
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! • : r^.'^iPlan I .. 



Plan XII* 



r 



1950 



1960' 

, ... 



1970 



gt L ^ 



1980 



1990 



;:r*Tigure 6" Ssmple Display of Historical and 
Possible Future Residential Dislocations 
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Figure 6 illustrates a very basic point in the development 
of information for consideration by decisionmakers. Too 
often the set of data we present has little meaning to those 
for whom it was developed simply because we fail to provide 
a basis for comparison. A simple device such as the his- 
torical trend line plot shown in figure 6 can provide the 
insight needed to intelligently absorb and appraise a 
multitude of data. 

Some Principles for Guiding the Selection of Evaluation 
Criteria 

Emerging from the foregoing discussion is a brief set of 
guiding principles, which can be considered in developing 
evaluation criteria. 

1. They should be developed on the basis of adopted regional 
and subregional goals and objectives. 

2. They should comprehensively cover all factors which decision- 
makers will use to evaluate and choose among alternative plans. 

3. They should have the explicit approval of a policy body. 

4. They should be capable of being measured and forecasted. 

5. They should be clearly and concisely developed and pre- 
senced, so as to be meaningful to professional, decision- 
maker, and layman alike. 
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SYSTEMS TO EVALUATE ("Let No Stone Go Unturned 11 ; 

While quite abviously no general rule can exist to govern the 
precise number and type of plan alternative to be examined, a 
few basic concepts can be applied to most situations. Normally 
the number of alternatives will depend upon the size and complexity 
of the urban region, the rate at which the region is growing, the 
requirements of decisionmakers and the public, and the financial 
and staff resources of the planning program. As a minimum, two 
systems which should be subject to evaluation, and which are 
frequently overlooked by many urban transportation studies, are 
(1) the existing system under current conditions, and (2) the 
existing (plus committed) system under future conditions. 

A. Evaluating the Existing System With Existing Travel 

Applying the full array of plan evaluation procedures to a 
base year system with base year travel is a step practiced 
by few transportation studies. Yet there are at least three 
basic advantages to performing such an exercise. 

1 # Appraising the Performance of the Existing System 

Performing an evaluation of base year conditions represents 
an excellent method for determining and reporting upon 
the existing performance characteristics of the regional 
transportation system. Measures such as accessibility, 
speed, volume to capacity ratios, and so on can be used 
to isolate current deficiencies, which can provide the 
factual basis need for an immediate action program. Such 
a program may be advocated by a transportation study well 
in advance of developing or updating the long range 
regional land use/ transportation plan. Thus, a useful 
service is provided, and the credibility of the entire 
planning process may be improved. 

2. Testing Evaluation Procedures 

The process of developing, testing, and evaluating a land 
use/ transportation plan may consume many months, if not 
years. The process can be streamlined to a certain extent 
by developing evaluation criteria, and an evaluation tech- 
nique in advance of their application to future systems, 
and testing the utility of these criteria and procedures 
by applying them to base year conditions. This enables 
the fine tuning of the evaluation system, and builds 
experience with and confidence in the procedures on the 
part of both the technical staff and the policy group. 
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3. Providing a Key Benchmark Against Which Future 
Alternatives May Be Compared 

A major problem with much of the planning information 
provided through evaluation of future alternatives lies 
in the difficulty in placing all this data within a 
comprehensible context. For example, a range in system 
speeds among future alternatives from 25 to 30 miles per 
hour may have little meaning unless current average 
speeds are known for the sake of comparison. Similarly, 
an annual consumption of 150 acres of land for transpor- 
tation purposes under a proposed alternative becomes a 
much more meaningful number if it is knowiwhat the average 
annual rate for the past few years has been. 

Thus, by comparing future alternatives, not only to each 
other, but to current conditions, a valuable frame of 
reference is provided facilitating more intelligent 
judgments. 

B. Evaluating the Existing (Plus Committed) System With Future 
Travel (Do- No thing Alternatives) 

Too often, planners fail to clearly indicate to policy 
officials and to the public, the consequences of completely 
halting the program of transportation improvements within 
an urban region. While such an action may appear unthinkable, 
recent experience with controversial transportation projects 
within urbanized areas has shown that the burden of demon- 
strating the need for a sustained program of capital improve- 
ment rests squarely with those government agencies and 
officials who have such responsibilities. Unless we can 
demonstrate the consequences of doing nothing, it will become 
increasingly difficult to do anything. 

C. Evaluating Future Alternative Plans 

While the evaluation of current conditions and the do-nothing 
alternative are to be strongly advised, the principal focus 
of the evaluation process will be the discrete number of land 
use/transportation alternatives which emanate from the plan 
development and testing phases. These alternative plans 
become the object of an intensive series of probing analyses 
which have been designed to evaluate the worthiness of each 
culminating in the selection and adoption of the most desirable 
one. 
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EVALUATION PROCEDURES ("Beware of Numbers - They May Not Lie, 

But They are Capable of Masking the Truth 1 



The principal objective in selecting an evaluation technique is 
to arrive. at a procedure which is capable of (1) incorporating 
a varied mix of seemingly incompatible evaluation criteria, 
(2) applying these criteria to the information gleaned from each 
plan, and (3) indicating which plan best meets the stated com- 
munity objectives. Quite a few techniques have evolved over 
recent years, and several of these will be described in the next 
paragraph. 

A. Plan Information Matrix 

The plan information matrix approach represents the most 
direct, and uncomplicated method of all. As shown in table 3, 
where the P.I.M. simply involves the listing of all previously 
established evaluation criteria, and the tabulation of all 
raw, unadjusted data associated with each plan, for each 
criterion. While this approach places on the policy group 
the burdens of distinguishing among criteria in terms of 
importance, and of absorbing a wealth of information for each 
of the many criteria, it has the advantage over the techniques 
subsequently described of avoiding the burying of key infor- 
mation behind a maze of weights, rates, ranks, and scores. 
Using the P.I.M. , therefore, facilitates the process of 
tradeoffs and compromise by dealing directly with data, in 
terms understandable by everyone, rather than dealing with 
the abstract numerical scores or rankings which characterize 
:;omo of the more esoteric procedures, 

B. Value Profile Method* 

The value profile method used in Twin Cities is another relatively 
simple approach to making evaluations based on numerous criteria. 
As shown in figure 7, value profiles can be drawn for various 
plans (in this case land use plans rather than transportation 
plans) showing their relationship to evaluation criteria which 
in many cases are not subject to quantitative measurement. 
This results in a graphical comparison of the extent to which 
the various goals are satisfied. In applying the method to 
transportation plan evaluation, evaluation criteria are repre- 
sented by value couplets composed of two reciprocal elements 
or criteria. One of the elements should correspond to satis- 
factory fulfillment of the goal, while the other would represent 
an undersirable situation. The value couplets are then arrayed 

* Much of the description of this and subsequent procedures 
draws directly from a summary of evaluation techniques 
prepared by David S. Gendell of the Urban Planning Division. 
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Table 3 

Sample Plan Information Matrix 



Sample 


Current System 


Plan Alternative 


Evaluation 
Criteria 


Performance 
or Recent Trend 


E + C 


A 


B 


C 


Annual 
Capital Cost 
million} 


25.0 


4.1 


21.6 


29.7 


38. 1 


Mean Effective 
Speed (m.p.h.) 


24.0 


17.0 


22.0 


25.0 


28.0 


100 million VMT 


3.1 


4.2 


3.7 


9 O 


2.7 


Dailv Transit 
Seat Miles per 
Thousand Capita 


33.0 


39.0 


37.0 


36.0 


36.0 


Average 

Annual Dislocations 
!• Residences 
2. Businesses 


25 
13 


8 
4 


15 
11 


29 
15 


87 
22 


Annual Taxable Land 
Consumption (Acres) 


36 


9 


31 


43 


62 


Plan Preference 
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Figure 7. — Value Profiles for Two of 
Twin Cities Alternative Land Use Pinns 
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within their goal area with the desirable "reciprocal 
criteria 11 on one side of the rating box and the undersirable 
ones on the other. The consensus concerning the ability of 
the plan to achieve the desirable reciprocal criteria is 
represented by a point, as 'shown in figure 7 t The points are 
then connected, and the result is a value profile for the 
plan. By plotting more than one profile on the same figure, 
graphical evaluations of alternative plans can be made. This 
method appears to have particular merit in presenting plan 
evaluation to laymen. One disadvantage of the value profile 
approach lies with the implicitly equal contribution of each 
criterion as it influences the position of the lines. 

C. Rank-Based Expected Value Method 

The rank-based expected value method was used by the Southeastern 
Wisconsin Regional Planning Commission to evaluate nonquanti- 
fiable as well as quantifiable elements of alternative land 
use plans. The method is also applicable to transportation 
plans. 

In this method, plans are ranked according to their ability to 
satisfy standards within a given objective. Additionally, the 
objectives are ranked in order of importance. In both instances, 
the most desirable situation is assigned the highest rank 
number. Since plans have different probabilities of implemen- 
tation, this element is brought into the analysis. This is 
accomplished through an adjustment to the plan score. 

Table 4 illustrates the method. The planning objectives 
(across the top of the table) are ranked in order of importance 
with the most important, "serve land use pattern and meet travel 
demand," being assigned a rank value of 3. The plans are then 
ranked on their ability to satisfy oach objective. Since this 
is a summary evaluation table, this derision is based on the 
plan's overall ability to satisfy the standards, and, therefore, 
objectives which make up the major objective groups. In the 
case of the second objective group, the controlled existing 
trend plan satisfies the component standards the best and is 
given a rank of 3. The rest of the ranks are then completed 
and the probability of implementation entered. The plan 
value or score is then obtained by multiplying the rank 
value or each planning objective by the rank value for the 
particular plan's ability to meet that objective and then sum- 
ming the similar values across the table. The resultant value 
is then multiplied by the probability of implementation. For 
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the controlled existing trend plan, this would give 

03x2) + (2x3) + .(1X3JJ0.6 - 9.0. Tne plan with the highest 

plan value is considered to be the best. 

In its application of the method, SEWRPC did not always 
assign the maximum value to the best plan. If none of the 
three plans satisfied a standard but one was better than 
the two, the best plan would be given a value of 2 and the 
other plans values of 1. Thus, a major drawback of the pro- 
cedure was partially overcome. This drawback is the fact 
than, in general, the plans are ranked rather than rated 
Thus, if three plans have values of 10, 9, and 2 for a par- 
ticular standard where high numbers are desirable, the plan 
with a value of 10 would be given a rank of 3, the one with 
9 a rank of 2, and the one with a value of 2 a rank of 1. 
The scale of the standard would be obscured. 

Serious question also exists concerning the use of the prob- 
ability of implementation factor at this point in the 
evaluation. For example, it can be seen from table 4 that 
even if the Satellite City plan were ranked highest in all 
categories, with a probability of implementation of 10 percent, 
its plan value would still be only 1.8 which is far below 
the next highest scoring plan. 

The Value Matrix 

The next method, the value matrix, is essentially an extension 
of the rank-based expected value method and has been applied 
in several areas. In this case, the goals, objectives, and 
evaluation criteria which will be used in plan evaluation are 
weighted to reflect their relative degree of importance. «The 
weights of the goals, objectives, and evaluation criteria 
should reflect those of the community or at least those of the 
community's decisionmakers. 

Once the goals, objectives, and evaluation criteria have been 
established and weighted, data are obtained concerning the rela- 
tion of the various plans to the individual evaluation criteria. 
This information can be in monetary terms, other quantif iables, 
or of a qualitative nature. The individual plans are next 
rated on a scale of 0 through 10 according to how well they 
satisfy the evaluation criteria and a summation made to pro- 
duce a plan score as it was in the rank-based expected value 
method. 
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A sample value matrix is shown in table 5. Across the top 
of the matrix are several goals which form the foundation 
for related objectives and evaluation criteria. Across the 
fourth row are the various weights developed for each 
evaluation criteria. To be valid, these weights must reflect 
the composite priorities of the group performing the 
evaluation. Under the weights are shown the data for each 
plan t Depending upon the data, each plan receives a rating 
from 0 to 10 (shown in bold face). For example, under the 
user cost criterion, Plan A, which has the highest user costs 
received a zero rating while Plans C and D which have the 
lowest user cost received a rating of 9. These rating are 
multiplied by the criteria weight of l t 3, with the products 
added to remaining products of rating and criteria weights, 
yielding the plan scores shown in the column on the far 
right. 

Items which are not included in the value matrix are considered 
either as contraints, or involve trade-offs between the plan 
score and the item under consideration. Plans which fail to 
meet constraints can be eliminated from further consideration. 

The constraints can be thought of as a means of overcoming the 
problem of using one particular value for the weight of each 
evaluation criteria regardless of the performance of the 
various plans. For example, an objective might be represented 
by the ^valuation criteria, number of acres of park land taken, 
and we might have several plans which take anywhere from no 
park land at all to all of the park land in the region. The 
problem is that the first few acres of park land taken may 
have a lower value to the community than the last few acres 
taken, yet a uniform value in terms of the weight of the eval- 
uation criteria is used for all plans. By using constraints, 
this problem of marginal utility is partially overcome by 
putting an infinite weight on those evaluation criteria that 
are not satisfied to at least the minimum value of the 
constraint. 

The other items considered outside of the value matrix involve 
the trade-offs with the plan score provided by the value matrix. 
An example of considerations handled in this manner would be 
the result of an economic evaluation or more simply the capital 
cost to construct the system. The decision as to which plan 
is best then boils down to one of cost-effectiveness in which 
,the construction cost is weighted against the effectiveness as 
measured by the plan score. 



26 - 



2 
I 

i 
I 



PLAN SCORE 


CM 


in 


£ 

t 


0D ! 
CD ! 


! «s ! 

re to i 
a * 

>° 


CO 

;% a 


8 

• 

Q 
M 

<: 


• 

CO 


Q 


CO 


Q 

H 
CO 






00 1 


9 

o 1 


i 

CO j 


c « 

1 i** 
1 w 


^ o 




i 






CO 


i; 






r 

<: 


CO 
M O 

r-H m 
o r-« 
* .• • . * 

J-5 !H 

^ CO 

w 


CO 

pd 


o 

* 

H 






VO 








VO 


to ' 


i 5 
i q 

1 o co 

1 M W 

, <5 2 

i f fs, 

i 


o 1 

M 

EH 

o 

§ 

xn 

M 
Q 


CO 
CO 

M Or, 
V} CO 


• 

o 


o 


o 




GO 


q 




H 




! < 5H 

; ^~ s-i 






















1 • J - " 
1 ' • ^ 

; ; oo 
! O W 

1 W Q 

! 

*' r - £ 


w 

M 


M Eh 
w5 CO 

Q 


w 

• 

o 


o 

1 


o 


1 


CD 


CO 

[ 




CO 


LO 

i 

l 


1 " 


1 § 






t 


i 




1 






i 






3 
1 « 


! M EH 

^\ 
► » 


! IT! t j 
\ > C 


1 

• 

H 


i 

i 5. 

. CO 

1 o 

I 




I 

o 

LTv 


in 


s 

ON 

m 


CD 


1 

i ^ 
i 


i CO; 

t 

1 




' r» rv 


! 






i 














♦ 












1 










!| 


j 


I 1 


o 








J 
I 
I 








ij 




i? 

il 

! 


• 

i 


! *— r 

I o ^ 

1 '0 
; :% >j 

» • r 


fx? M 

«• ■ - ■ 

m n 
» ♦ • ■ 


A o 
> a 

1 


ON 
• 

H 


is 

CO 

I 

i 


CO 


o 

n 




H 




!o 

•\ 

t 

H 

I 


i 
j 


• ^ 


1 r_< 
< 


• o r> 

1 

t 


• 

H 


i 

1 

i l 


o 


1 
i 






CD 


O 
VO 
H 


i 




i 

i • "i 


: 8. 

M 1 


H 


»!/ 


.j 


!.«' 


: fl 


< • 


• t: 

ri 


t m 

l< 

n 


• i i 
;i 


f • 

>♦ i 

( 1 




i ■•■» 
\ ■' 


i ^ 
1 > 

• W 




1 

1 


, . rx. 

ll 
l) 

;i 




i! 

!i 






•i 
i 
i 

It 





9 

ERIC 



- 27 - 

CONCLUSION ("The Measure of Success is Not in the Doing, 
But in the Deeds 11 ) 

The evaluation of alternative long range plans represents one 
of the principal focal points in a continuing, comprehensive 
land use/transportation planning effort. Into this key phase 
of the planning process flow the numerous elements which 
characterize a long range regional planning program: data 
collection, formulation of goals, objectives, and evaluation c 
criteria, development of land use and travel models, forecasts 
of the magnitude and distribution of urban activities, formu- 
lation of land use and transportation system alternatives, and 
the development of travel projections by purpose, mode, time, and 
orientation. Out of this, vast collection of information must 
emerge those items of data necessary for a group of normally 
nontechnical, policy oriented individuals to render intelligent 
judgments concerning the relative mertis of plan alternatives. 

To succeed this process requires a clear definition of regional 
goals and objectives, translated into meaningful, and acceptable 
evaluation criteria. It requires en understanding and acceptance 
on the part of the policy group and the technical staff as to 
their relative roles and responsibilities. And it requires the 
use of procedures which are capable of providing the information 
required by decisionmakers, in terms clear and understandable 
to them. However, unlike more technically oriented aspects of the 
planning process, there is no inherently correct method for per- 
forming plan evaluation. The effectiveness of the evaluation 
effort depends not so much on how it is done, but on what it has 
produced. If it produces an adopted long range plan and short 
range program which are generally accepted by the community, 
which serve as the basis for decisions of regional importance 
concerning transportation improvements and urban development, 
then it may be safely assumed that the evaluation process has 
succeeded. On the other hand, regardless of how sophisticated 
the approach, if the plan and program which are ultimately 
recommended merely serve as a facade, and actually have relatively 
little effect on major decisions of the type just described, then 
the effort must be characterized as less than successful. 



